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Features  of  Software  Development  Tools 
Raymond  C.   Houghton,  Jr. 


Abstract 

Software  tools  are  powerful  productivity  and  quality 
aids  that  in  many  cases  are  not  being  used  effectively. 
This  report  discusses  an  effort  to  lessen  this  problem  by 
providing  a  formal  way  in  which  tools  can  be  classified 
according  to  the  features  that  they  provide. 

Key  words:      Dynamic   analysis;    programming   aids;    software  de- 
velopment;   software    engineering;    software    tools;  static 
analysis;  taxonomy. 


1.  Introduction 

Software  development  tools  are  computer  programs  that 
aid  the  specification,  construction,  testing,  analysis, 
management,  documentation,  and  maintenance  of  other  computer 
programs.  Thus,  software  development  tools  include  the 
traditional  tools  of  a  programmer  (e.g.  compilers, 
editors),  more  recently  developed  tools  (e.g.  design  aids, 
program  analyzers,  testing  tools) ,  and  tools  currently  in 
the  research  stage  (e.g.  formal  verifiers,  programming 
environments) .  Tools  are  important  because  they  can  be  used 
to  increase  software  productivity  and  quality.  As  a  result, 
their  use  has  evolved  as  an  important  part  of  software 
development.  Most  importantly,  they  represent  a  class  of 
software  that  can  be  used  and  reused  within  many  different 
software  development  environments.  The  use  of  software 
tools  provides  the  opportunity  to  reduce  cost  and  improve 
competitive  advantage. 

Although  software  tools  are  an  important  part  of 
quality  software  development,  there  does  not  currently  exist 
a  clear  body  of  techniques  for  making  effective  use  of  such 
tools.       This     situation  exists  in  part  due  to  the  confusion 


NOTE:  Certain  commercial  products  are  identified  in  this 
paper  for  clarification  of  specific  concepts.  In  no  case 
does  such  identification  imply  recommendation  or  endorsement 
by  the  National  Bureau  of  Standards,  nor  does  it  imply  that 
the  material  identified  is  necessarily  the  best  for  the 
purpose  o 
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generated  by  the  lack  of  standard  terminology  for  the 
functions  and  characteristics  of  software  tools.  It  is 
difficult  to  compare  and  to  evaluate  alternative  tool 
packages  for  use  in  specific  operating  environments.  As  a 
result,  there  is  a  lack  of  technology  transfer  by  tool 
developers  to  the  user  community. 

This  report  discusses  an  approach  aimed  at  correcting 
some  of  these  problems.  It  presents  a  framework  for  tool 
comparison  and  a  scheme  for  classifying  tools  by  the 
features  they  provide.  Features  of  software  tools  are 
presented  in  a  hierarchical  classification  system  called  a 
taxonomy.  Following  a  period  of  comment  and  review,  NBS 
plans  to  issue  the  taxonomy  as  a  Federal  Information 
Processing  Standards  Publication  (FIPS-PUB)  Guideline  to 
provide  a  common  reference  within  the  Federal  Government  for 
terms  and  definitions  associated  with  software  development 
tool s . 


2.     The  Importance  of  Tool  Features 

Software  development  tools  have  grown  in  complexity 
over  recent  years  as  have  other  software  applications.  Most 
early  tools  performed  a  single  function  and  were  very  simple 
to  operate.  These  early,  simpler  tools  have  since  given  way 
to  more  complex  tools  which  in  some  cases  have  their  own 
built-in  command  languages.  This  growth  is  leading  to  tool 
systems  where  the  programmer  works  totally  within  the  system 
to  develop  software. 

This  evolution  of  t®ol  development  has  caused  major 
problems  in  communicating  exactly  what  a  tool  does.  For 
example,  a  compiler  once  was  a  simple  tool  that  performed 
the  single  function  of  translating  high  level  language  to 
machine  language.  Today,  we  have  compilers  that  interpret, 
optimize,  perform  run-time  checks,  and  generate 
cross-references,  profiles,  formatted  output,  and  other 
forms  of  documentation.  Does  the  term  compiler  adequately 
describe  these  multi-functional  systems,  or  do  people  still 
think  of  a  compiler  as  a  simple  tool  that  performs  a  single 
function?  There  are  other  examples  of  tools  whose 
complexity  has  outgrown  their  names.  These  include 
verification  systems,  program  analyzers,  program 
preprocessors,  flow  analyzers,  software  development  systems, 
simulators,  and  programming  environments.  In  each  of  these 
examples,  one  should  press  for  further  details  about  the 
tool  to  determine  exactly  what  it  does. 
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The  features  incorporated  in  a  software  tool  are  the 
details  one  should  seek.  Careful  identification  of  features 
allows  one  to  distinguish  one  tool  from  another  and  to 
determine  which  is  more  appropriate  for  a  given  application. 
In  order  to  do  this,  one  must  acquire  detailed  information 
on  a  tool.  This  should  include  what  a  tool  accepts  as  input 
and  how  it  accepts  it,  the  way  it  manipulates  and  analyzes 
that  input,  and  what  a  tool  produces  as  output  for  both  the 
tool  user  and  for  further  processing  by  other  tools.  With  a 
careful  analysis  of  this  information,  one  can  begin  to 
understand  the  real  capabilities  of  a  tool  and  can  compare 
these  capabilities  with  those  of  other  tools.  The  taxonomy 
of  tool  features  presented  in  Section  4  provides  a  framework 
for  communicating  and  a  basis  for  understanding  the 
capabilities  of  a  tool. 


3.     Formalizing   the  Use  of  loci  Features 

For  now  let  us  assume  that  we  have  a  way  to  carefully 
and  succinctly  discover  the  individual  features  of  a 
software  tool.  The  sections  that  follow  will  address  how 
one  could  use  the  resulting  information.  Examples  of  each 
of  these  uses  will  be  shown  in  Section  5. 

Compa  r  i  son  _&  Eval  ua  t  ion  o  f  So  f  twa  r  e  Tool  s  .  Once  all  the 
features  of  a  tool  are  known,  its  features  can  be  compared 
to  the  features  of  other  tools.  The  presence  (or  absence) 
of  a  specific  feature  in  a  given  tool  is  apparent.  Thus,  a 
potential  tool  user  has  a  rational  means  for  determining 
whether  to  include  a  tool  for  consideration.  For  example, 
if  a  tool  user  is  interested  in  a  tool  that  has  the  features 
of  high  level  language  input,  program  instrumentation, 
dynamic  coverage  analysis,  and  output  report  generation, 
then  the  user  may  find  several  tools  that  fall  within  this 
classification.  The  user  can  then  focus  on  the  additional 
features  offered  by  each  of  the  tools  and  perform  a  cost 
benefit  analysis  on  these  additional  features.  Once  a  set 
of  candidate  tools  has  been  determined  based  on  desired 
features,  the  user  can  focus  on  other  considerations,  such 
as  implementation  languages,  application  language,  hardware 
and  software  requirements,  availability,  performance, 
portability,  support  and  available  documentation. 

Basis  f o r  Info rmat ion  Organ i  za t ion .  Since  features  provide 
a  vehicle  for  the  comparison  of  software  tools,  one  of  their 
apparent  uses  is  as  a  basis  for  retrieval  in  a  database. 
Just  as  library  searches  are  based  on  keywords,  tool 
searches  can  be  based  on  features.  Relationships  can  be 
formed  between  the  features  of  a  tool  and  its  name.  Thus, 
when  one  wants  to  know  all  the     tools     available     that  have 
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certain  features,  then  an  additional  relationship  may  be 
defined  that  relates  these  features  only  with  those  tools 
that  possess  them.  From  here  it  is  a  simple  matter  to 
provide  any  other  information  that  may  be  desired  about  the 
tools  in  question. 

The  NBS  Software  Tools  Database  [Houg80]  is  a 
relational  database  that  possesses  this  capability.  One  of 
the  purposes  of  the  database  is  to  initiate  a  means  by  which 
persons  in  the  Federal  Government  can  determine  what  tools 
are  available  and  what  their  capabilities  are.  An  example 
of  tool  information  retrieval  from  the  database  is  shown  in 
Section  5. 

CI assi f ica t ion  o f  Software  Tool s .  Since  features  provide  a 
means  by  which  we  can  describe  software  tools,  then  they  can 
also  provide  a  means  by  which  we  can  classify  software 
tools.  In  order  to  do  this,  we  must  discover  and  describe 
all  the  features  for  every  known  software  development  tool, 
gather  them  all  together,  eliminate  duplication,  and  order 
them.  The  resulting  order  is  called  a  taxonomy.  The 
taxonomy  is  used  to  classify  tools  by  associating  the 
ordered  features  with  the  tool  being  classified.  An  example 
of  this  will  be  given  in  section  5  after  first  presenting 
the  taxonomy. 


4.     A  Taxonomy  of  Tool  Features 

The  taxonomy  that  will  be  discussed     was     developed  to 
satisfy  the  following  goals: 

a.  The  taxonomy  shall  allow  NBS  to  classify  all  currently 
available  software  development  tools. 

b.  Each  class  shall   include     terms     that     have     the  most 
specific  meaning  where  possible, 

c.  The  classes  shall  allow  easy  comparison  and  evaluation 
of  tools. 

d.  The  classes  shall  be  related  to  the  life  cycle     of  an 
automated  data  system. 

All  of  the  goals  were  met  except  for  the  last.  Life  cycle 
relationships,  although  possible,  were  found  to  be  outside 
the  scope  of  the  taxonomy  that  was  developed. 

The  initial  development  of  the  taxonomy    was  performed 
under     contract     [1]    to  the  National  Bureau  of  Standards  and 


[1]  Contract  NB79SBCA0273  to  SoHar,  Inc.  and  SoHaR 
Subcontract  No.     102  to  Software  Management  Consultants. 
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is  reported  in  detail  in  [Reif80].  The  present  taxonomy  is 
an  update  of  the  one  presented  in  that  report.  Further 
updates  are  anticipated  as  the  taxonomy  approaches  adoption 
as  a  FIPS.  Therefore,  comments  on  the  usefulness, 
implementation,  structure,  and  content  are  solicited  by  the 
autho r  . 

The  taxonomy  is  a  hierarchical  order  of  software  tool 
features  and  is  illustrated  in  figure  1.  At  the  bottom  or 
feature  level  of  the  hierarchy  are  a  total  of  52  tool 
features.  Each  of  these  features  will  be  defined  and 
discussed  in  the  sections  that  follow.  At  the  third  level 
are  the  classes  of  tool  features.  These  are  subject, 
control  input,  transformation,  static  analysis,  dynamic 
analysis,  machine  output,  and  user  output.  Each  of  the  52 
features  have  been  placed  into  one  of  these  classes.  In 
order  to  provide  a  way  to  abbreviate  a  tool  classification, 
keys  have  been  assigned  to  each  of  the  elements  in  the  third 
and  fourth  levels.  Each  of  the  keys  will  be  identified  in 
the  sections  that  follow  and  examples  of  their  use  will  be 
shown  in  Section  5. 

The  second  level  of  the  taxonomy  covers  the  basic 
processes  of  a  tool.  These  are  input,  function,  and  output. 
The  highest  level,  software  development  tool  features, 
covers  all  the  levels  below.  The  taxonomy  has  been  designed 
to  be  expandable.  It  is  expected  that  future  updates  of  the 
taxonomy  will  include  additional  features  and  possibly 
additional  levels. 
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Figure  1.       Taxonomy  of  Software  Development  Tool  Features 
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Input .  Input  features  are  based  on  the  forms  of  input  which 
can  be  provided  to  a  tool.  These  features  fall  into  two 
classes,  one  which  is  based  on  how  the  tool  should  operate 
(control  input)  and  the  other  based  on  what  the  tool  should 
operate  on  (subject)  .  Using  a  compiler  as  an  example,.  the 
subject  is  the  code  that  will  be  compiled  and  the  control 
input  is  the  set  of  commands  which  specify  compiler  options 
(optimize,  debug,  cross  reference,  etc.). 


INPUT 

Subject  Control  Input 

II;     Text  CI.  Commands 

12.  VHLL  C2.  Parameters 

13.  Code 

14.  Data 

Table  1.  Input 


Sub j  ec  t  (Key ;  I)  .  The  subject  is  usually  the  main 
input  to  a  tool.  It  is  the  input  which  is  subjected  to 
the  main  functions  performed  by  a  tool.  The  four  types 
of  subjects  are  data,  code,  VHLL  (very  high  level 
language)  ,  and  text. 

11.  Text  -  The  input  to  the  tool  is  presented  in  a 
natural  language  form.  Certain  types  of  tools  are 
designed  to  operate  on  text  only  (e.g.,  text 
editors,  document  preparation  systems)  and  require 
no  other  input  except  directives  or  commands. 

12.  VHLL  -  The  input  to  the  tool  is  a  program 
written  in  a  very  high  level  language  that  is 
typically  not  executed.  Three  recognized  types  of 
VHLL's  are  requirements  languages,  design 
languages,  and  description  languages. 
Requirements  and  design  languages  are  both  used  to 
define  programs  and  to  provide  a  means  for 
generating  automatic  documentation.  Examples  of 
requirements  languages  include  the  Problem 
Statement  Language  [Teic77]  and  the  Requirements 
Statement  Language  [Bell77].  An  example  of  a 
design  language  is  Program  Design  Language 
[Cain75].  Description  languages  are  used  to 
describe  attributes  of  the  input  in  high-level, 
non-procedural  form.  An  example  of  a  description 
language  is  Backus-Naur  Form   (BNF) . 
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13.  Code  -  The  input  to  the  tool  is  a  program 
written  in  a  language  that  is  subject  to  a  given 
translation  process  before  it  is  executed.  Code 
is  the  language  form  in  which  most  programming 
solutions  are  expressed  and  includes  high  level 
languages,  assembly  languages,  object 
representations  or  parametric  representations. 
High  level  languages  are  problem-oriented  (e.g., 
COBOL  and  FORTRAN) ,  while  assembly  languages  are 
machine  oriented  but  symbolic  in  nature.  Object 
code  represents  an  input  that  is  typically 
executable  without  further  translation,  while 
parametric  representation  (e.g.,  NAMELIST  in 
FORTRAN  IV)    inputs  variables  to  be  translated. 

14.  Data  -  The  input  to  the  tool  is  a  string  of 
characters  to  which  meaning  is  or  might  be 
assigned.  The  input  (e.g.  raw  data)  is  not  in  an 
easily  interpreted,  natural  language  form.  A 
simulator  that  accepts  numeric  data  to  initialize 
its  program  variables  is  an  example  of  a  tool  that 
has  data  as  input. 

Some  tools,  such  as  editors,  operate  on  any  if  the  four 
of  these  input  forms.  In  cases  such  as  this,  the  input 
form  is  chosen  from  the  viewpoint  of  the  tool.  Since 
editors  view  the  input  form  as  text,  then  text  would  be 
the  correct  choice  for  this  tool. 

Control  Input  (Key ;  C) .  Control  inputs  specify  the 
type  of  operation  and  the  detail  associated  with  an 
operation.  They  describe  any  separable  commands  that 
are  entered  as  part  of  the  input  stream. 

CI.  Command  s  -  The  control  input  to  the  tool 
consists  primarily  of  procedural  operators,  each 
capable  of  invoking  a  system  function  to  be 
executed.  A  directive  invoking  a  series  of 
diagnostic  commands  (i.e.,  TRACE,  DUMP,  etc.)  at 
selected  breakpoints  is  an  example.  A  tool  that 
performs  a  single  function  will  not  have  this 
feature  but  will  most  likely  have  the  next. 

C2.  Parameters  -  The  control  input  is  a  series  of 
parameters  that  are  associated  with  the  functions 
performed  by  the  tool.  Parameters  are  usually 
entered  as  a  result  of  a  prompt  from  a  tool  or  may 
be  embedded  in  the  tool  input.  An  interactive 
trace  routine  that  prompts  for  breakpoints  is  an 
example  of  a  tool   with  parametric  input. 
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Function .  The  features  for  this  class  are  shown  in  Table  2. 
They  describe  the  processing  functions  performed  by  a  tool 
and  fall  into  three  classes:  transformation,  static 
analysis,  and  dynamic  analysis.  Again  using  a  compiler  as 
an  example,  the  transformation  features  would  be  translation 
and  possibly  optimization,  the  static  analysis  features 
would  be  error  checking  and  possibly  cross  reference,  and  a 
dynamic  analysis  feature  may  be  tracing. 


Transformation 


Tl. 

Editing 

SI. 

T2. 

Formatting 

S2. 

T3. 

Instrumentation 

S3. 

T4. 

Opt  im  i  zat  ion 

S4. 

T5. 

Restr uctur  ing 

S5. 

T6. 

Translation 

S6. 

S7. 

S8. 

S9. 

S10 

Sll 

S12 

S13 

S14 

S15 

S16 

S17 

818 

S19 

FUNCTION 

Static  Analysis 

Aud  iting 
Comparison 

Complexity  Measurement 
Completeness  Checking 
Consistency  Checking 
Cost  Estimation 
Cross  Reference 
Data  Flow  Analysis 
Error  Checking 

.   Interface  Analysis 

.  Management 

.   Resource  Estimation 

.  Scanning 

.  Scheduling 

.   Statistical  Analysis 

.   Structure  Checking 

.  Tracking 

.   Type  Analysis 

.    Units  Analysis 


Dynamic  Analysis 

Dl.  Assertion  Checking 

D2.  Constraint  Evaluation 

D3.  Coverage  Analysis 

D4.  Resource  Utilization 

D5.  Simulation 

D6.  Symbolic  Execution 

D7.  Timing 

D8.  Tracing 

D9 .  Tuning 


Table  2.  Function 
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Transformation  (Key ;  T) .  Transformation  features  describe 
how  the  subject  is  manipulated  to  accommodate  the  user's 
needs.  They  describe  what  transformations  take  place  as  the 
input  to  the  tool  is  processed.  There  are  six 
transformation  features.  Each  of  these  features  is  briefly 
defined  as  follows: 

Tl.  Ed i ting  -  modifying  the  context  of  the  input  by 
inserting,  deleting,  or  moving  characters, 
numbers,  or  data. 

T2.  Formatting  -  arranging  a  program  according  to 
predefined  or  user  defined  conventions.  A  tool 
that  "cleans  up"  a  program  by  making  all  statement 
numbers  sequential,  alphabetizing  variable 
declarations,  indenting  statements,  and  making 
other  standardizing  changes  has  this  feature. 

T3.  Instrumentation  -  adding  sensors  and  counters  to 
a  program  for  the  purpose  of  collecting 
information  useful  for  dynamic  analysis  [Paig74]. 
Most  code  analyzers  instrument  the  source  code  at 
strategic  points  in  the  program  in  order  to 
collect  execution  statistics  required  for  coverage 
analysis  and  tuning   (see  D2  and  D9) . 

T4.     Opt im i za t ion  -  modifying  a  program     to  improve 

performance,     e.g.       to     make  it  run  faster  or  to 

make  it  use  fewer  resources.  Optimizing  compilers 
have  this  feature. 

T5.  Restr uctur  ing  -  reconstructing  and  arranging  the 
subject  in  a  new  form  according  to  well-defined 
rules.  A  tool  that  converts  unstructured  code 
into  structured  code  is  an  example  of  a  tool  with 
this  feature. 

T6.  Transl a t ion  -  to  convert  from  one  language  form 
to  another.  Tools  that  have  this  feature  include 
compilers,  structured  language  preprocessors,  and 
conversion  tools. 

b.     Static  Anal ysis   (Key :     S) .  Static  analysis 

features  specify  operations  on  the  subject  without 
regard  to  the  executab il i ty  of  the  subject  [Howd78]  . 
They  describe  the  manner  in  which  the  subject  is 
analyzed.  There  are  nineteen  static  analysis  features. 
Each  is  briefly  described  as  follows: 

SI.  Aud  i  t ing  -  conducting  an  examination  to 
determine  whether  or  not  predefined  rules  have 
been  followed.     A  tool   that     examines     the  source 
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code  to  determine  whether  or  not  coding  standards 
are  complied  with  is  an  example  of  a  tool  with 
this  feature. 

52.  Compa r i son  -  Assessing  similarities  between  two 
or  more  items.  A  tool  that  compares  programs  or 
test  runs  for  maintaining  version  control  has  this 
feature . 

53.  Compl ex  i ty  Measurement  -  Determining  how 
complicated  an  entity  (e.g.,  routine,  program, 
system,  etc.)  is  by  evaluating  some  number  of 
associated  characteristics  [Mcca76]  [Hals77].  For 
example,  the  following  characteristics  can  impact 
complexity:  instruction  mix,  data  references, 
structure/  control  flow,  number  of  interactions/ 
interconnections,  size,  and  number  of 
computations . 

54.  Compl eteness  Checki ng  -  assessing  whether  or  not 
an  entity  has  all  its  parts  present  and  if  those 
parts  are  fully  developed  [Boeh78].  A  tool  that 
examines  the  source  code  for  missing  parameter 
values  has  this  feature. 

55.  Consi stency  Checki  ng  -  determining  whether  or 
not  an  entity  is  internally  consistent  in  the 
sense  that  it  contains  uniform  notation  and 
terminology  [Walt78],  or  is  consistent  with  its 
specification  [Robi77].  Tools  that  check  for 
consistent  usage  of  variable  names  or  tools  that 
check  for  consistency  between  design 
specifications  and  code  are  examples  of  tools  with 
this  feature. 

56.  Cost  Est imat ion  -  assessing  the  behavior  of  the 
variables  which  impact  life  cycle  cost.  A  tool  to 
estimate  project  cost  and  investigate  its 
sensitivity  to  parameter  changes  has  this  feature. 

57.  Cross  Reference  -  referencing  entities  to  other 
entities  by  logical  means.  Tools  that  generate 
call  graphs  or  tools  that  identify  all  variable 
references  in  a  subprogram  have  this  feature. 

58.  Data  Flow  Anal ys i s  -  graphical  analysis  of  the 
sequential  patterns  of  definitions  and  references 
of  data  [Oste76] .  Tools  that  identify  undefined 
variables  on  certain  paths  in  a  program  have  this 
feature. 
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S9.  Er  ro  r  Checki  ng  -  determining  discrepancies, 
their  importance,  and/or  their  cause.  Tools  used 
to  identify  possible  program  errors,  such  as 
misspelled  variable  names,  arrays  out  of  bounds, 
and  modifications  of  a  loop  index  are  examples  of 
tools  with  this  feature, 

510.  In te r face  Anal ys i s  -  checking  the  interfaces 
between  program  elements  for  consistency  and 
adherence  to  predefined  rules  and/or  axioms.  A 
tool  that  examines  interfaces  between  modules  to 
determine  if  axiomatic  rules  for  data  exchange 
were  obeyed  has  this  feature. 

511.  Manag ement  -  aiding  the  management  or  control 
of  software  development.  Tools  that  control 
access,  updates,  and  retrievals  of  software; 
tools  that  maintain  and  control  data  definition 
and  use;  and  tools  that  manage  test  data  sets  are 
examples  of  tools  with  this  feature. 

512.  Reso urce  Estimation  -  estimating  the  resources 
attributed  to  an  entity.  Tools  that  estimate 
whether  or  not  memory  limits,  input/output 
capacity,  or  throughput  constraints  are  being 
exceeded  have  this  feature. 

513.  Scanning  -  examining  an  entity  sequentially  to 
identify  key  areas  or  structure.  A  tool  that 
examines  source  code  and  extracts  key  information 
for  generating  documentation  is  an  example  of  a 
tool  with  this  feature. 

514.  Sched ul ing  -  assessing  the  schedule  attributed 
to  an  entity.  A  tool  that  examines  the  project 
schedule  to  determine  its  critical  path  (shortest 
time  to  complete)    has  this  feature. 

515.  Statistical  Anal ys i s  -  performing  statistical 
data  collection  and  analysis.  A  tool  that  uses 
statistical  test  models  to  identify  where 
programmers  should  concentrate  their  testing  is 
one  example.  A  tool  that  tallies  occurrences  of 
statement  types  is  another  example  of  a  tool  with 
this  feature. 

516.  Structure  Checki  ng  -  detecting  structural  flaws 
within  a  program  (e.g.  improper  loop  nestings, 
unreferenced  labels,  unreachable  statements,  and 
statements  with  no  successors) . 
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517.  Tracki  ng  -  tracking  the  development  of  an 
entity  through  the  software  life  cycle.  Tools 
used  to  trace  requirements  from  their 
specification  to  their  implementation  in  code  have 
this  feature. 

518.  Type  Anal ys i s  -  evaluating  whether  or  not  the 
domain  of  values  attributed  to  an  entity  are 
properly  and  consistently  defined.  A  tool  that 
type  checks  variables  has  this  feature. 

519.  Uni ts  Anal ys i s  -  determining  whether  or  not  the 
units  or  physical  dimensions  attributed  to  an 
entity  are  properly  defined  and  consistently  used. 
A  tool  that  can  check  a  program  to  ensure 
variables  used  in  computations  have  proper  units 
(e.g.  hertz  =  cycl es/ seconds)  is  an  example  of  a 
tool  with  this  feature. 

Dynam  ic  Anal ys  i  s   (Key ;     D)  .  Dynamic  analysis 

features  specify  operations  that  are  determined  during 
or  after  execution  takes  place  [Howda78].  Dynamic 
analysis  features  differ  from  those  classified  as 
static  by  virtue  of  the  fact  that  they  require  some 
form  of  symbolic  or  machine  execution.  They  describe 
the  techniques  used  by  the  tool  to  derive  meaningful 
information  about  a  program's  execution  behavior. 
There  are  nine  dynamic  analysis  features.  Each  is 
briefly  described  as  follows: 

Dl.  Assertion  Checki ng  -  checking  of  user-embedded 
statements  that  assert  relationships  between 
elements  of  a  program.  An  assertion  is  a  logical 
expression  that  specifies  a  condition  or  relation 
among  the  program  variables.  Checking  may  be 
performed  with  symbolic  or  run-time  data.  Tools 
that  test  the  validity  of  assertions  as  the 
program  is  executing  or  tools  that  perform  formal 
verification  of  assertions  have  this  feature. 

D2.  Constra  int  Eval ua  tion  -  generating  and/or 
solving  path  input  constraints  for  determining 
test  input  [Clar76] .  Tools  that  assist  the 
generation  of  or  automatically  generate  test  data 
have  this  feature. 

D3.  Cover ag e  Anal ys i s  -  determining  and  assessing 
measures  associated  with  the  invocation  of  program 
structural  elements  to  determine  the  adequacy  of  a 
test  run  [Fair78] .  Coverage  analysis  is  useful 
when  attempting  to  execute  each  statement,  branch, 
path,     or     iterative     structure   (i.e.,   DO  loops  in 
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FORTRAN)  in  a  program.  Tools  that  capture  this 
data  and  provide  reports  summarizing  relevant 
information  have  this  feature. 

D4.  Reso  ur ce  Ut  i 1 i  za  t  ion  -  analysis  of  resource 
utilization  associated  with  system  hardware  or 
software.  A  tool  that  provides  detailed  run-time 
statistics  on  core  usage,  disk  usage,  queue 
lengths,  etc.  is  an  example  of  a  tool  with  this 
feature . 

D5.  Simulation  -  representing  certain  features  of 
the  behavior  of  a  physical  or  abstract  system  by 
means  of  operations  performed  by  a  computer.  A 
tool  that  simulates  the  environment  under  which 
operational  programs  will  run  has  this  feature. 

D6.  Symbol ic  Exec  ut  ion  -  reconstructing  logic  and 
computations  along  a  program  path  by  executing  the 
path  with  symbolic  rather  than  actual  values  of 
data   [Darr78] . 

D7.  Tim ing  -  reporting  actual  CPU  times  associated 
with  parts  of  a  program. 

D8.  Trac ing  -  tracking  the  historical  record  of 
execution  of  a  program.  Tools  that  produce  trace 
histories  or  allow  the  setting  of  breakpoints  for 
tracking  down  errors  have  this  feature. 

D9.  Tun  ing  -  determining  what  parts  of  a  program  are 
being  executed  the  most.  A  tool  that  instruments 
a  program  to  obtain  execution  frequencies  of 
statements  is  a  tool  with  this  feature. 


Output .  Output  features,  which  provide  the  link  from  the 
tool  to  the  user,  are  illustrated  in  Table  3.  They  describe 
what  type  of  output  the  tool  produces  for  both  the  human 
user  and  the  target  machine  (where  applicable) .  Again  using 
a  compiler  as  an  example,  the  user  output  would  be 
diagnostics  and  possibly  listings  and  tables  (cross 
reference) ,  and  the  machine  output  would  be  object  code  or 
possibly  intermediate  code. 
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OUTPUT 


User  Output 


Machine  Output 


Ul. 
U2. 
U3. 
U4. 
U5. 
U6. 


Computational  Results  Ml. 
Diagnostics  M2, 
Graphics  M3. 
Listings  M4. 
Text  M5. 
Tables  M6. 


Data 

Intermediate  Code 
Object  Code 
Prompts 
Source  Code 
Text 


Table  3. 


Output 


User  Output  (Key ;  U) .  User  output  features  handle 
the  interface  from  the  tool  to  the  human  user.  They 
describe  the  types  of  information  that  the  tool 
provides  for  the  user  and  the  form  in  which  this  output 
is  presented.  There  are  six  user  output  features. 
Each  is  briefly  defined  as  follows: 

Ul.  Computational  Resul ts  -  an  output  from  the  tool 
is  simply  the  result  of  a  computation.  The  output 
is  not  in  an  easily  interpreted  natural  language 
f o rm   (e.g.  text). 

U2.  Diagnostics  -  an  output  from  the  tool  simply 
indicates  that  a  software  discrepancy  has 
occurred.  An  error  flag  from  a  compiler  is  an 
example . 

U3.  Graphics  -  an  output  from  the  tool  is 
graphically  presented  with  symbols  indicating 
operations,  flow,  etc.  A  tool  providing  a 
flowchart  of  a  program  is  an  example. 

U4.  Listings  -  an  output  from  the  tool  is  a  listing 
of  a  source  program  or  data  and  may  be  annotated. 
Many  different  forms  of  listings  can  be  generated. 
Some  may  be  user  controlled  through  directives. 

U5.  Text  -  an  output  from  the  tool  is  in  a  natural 
language  form.  The  output  may  be  a  choice  of  many 
different  types  of  reports  and  the  formats  may  be 
user  defined. 

U6.  Tables  -  an  output  from  the  tool  is  arranged  in 
parallel  columns  to  exhibit  a  set  of  facts  or 
relations  in  a  definite,  compact  and  comprehensive 
form.  A  tool  that  produces  a  decision  table 
identifying  a  program's  logic  (conditions, 
actions,       and       rules       that     are     the     basis  of 
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decisions)    is  an  example. 

b.  Machine  Output  (Key ;  M) .  Machine  output  features 
handle  the  interface  from  the  tool  to  a  target  machine. 
They  describe  what  the  machine  expects  to  see  as  output 
from  the  tool.  There  are  six  machine  output  features. 
Each  is  briefly  described  as  follows: 

Ml.  Data  -  the  input  to  the  machine  is 
representations  of  characters  or  numeric 
quantities  to  which  meaning  has  been  assigned.  A 
tool  generating   input  to  a  plotter  is  an  example. 

M2.  Intermed  ia te  Code  -  the  input  to  the  machine  is 
between  source  code  and  machine  code.  A  tool 
producing  P-code  for  direct  machine  interpretation 
is  an  example. 

M3.  Ob j ec t  Code  -  the  input  to  the  machine  is  a 
program  expressed  in  machine  language  which  is 
normally  an  output  of  a  given  translation  process. 
A  tool  producing  relocatable  load  modules  for 
subsequent  execution  is  an  example. 

M4.  Prompts  -  the  input  to  the  machine  is  a  series 
of  procedural  operators  that  are  used  to 
interactively  inform  the  system  in  which  the  tool 
operates  that  it  is  ready  for  the  next  input. 

M5.  Source  Code  -  the  input  to  the  machine  is  the 
program  written  in  a  procedural  language  that  must 
be  input  to  a  translation  process  before  execution 
can  take  place. 

M6.  Text  -  the  input  to  the  machine  is  presented  in 
a  written  form  that  can  be  read  without  machine 
interpretation.  A  tool  producing  English  text 
which  is  fed  to  the  machine  is  an  example. 


5,     Using  the  Taxonomy 

Section  3  discussed  in  a  very  general  level  the  use  of 
tool  features.  Now  that  we  have  defined  the  features  of  a 
tool  and  given  each  of  them  individual  keys,  we  can  discuss 
some  very  specific  uses  of  the  taxonomy. 

Tool  Classi f icat ion .  To  classify  tools  with  the  taxonomy, 
one  must  identify  a  tool's  input,  functional,  and  output 
features.  As  many  features  are  identified  as  necessary  to 
fully    describe     the  capability  of  a  tool.     Identifying  tool 
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features,  in  some  cases,  can  be  a  major  challenge  because 
most  tool  descriptions  fail  to  provide  sufficient 
information.  Considerable  effort  may  be  required  to  acquire 
the  information  necessary  to  identify  what  the  tool  does  and 
how  it  interfaces  with  the  external  environment. 

The  result  of  classification  is  a  features  designator 
called  the  taxonomy  key.  The  key  is  formed  by  collecting 
the  individual  feature  keys  according  to  the  their  position 
in  the  taxonomy.  For  example,  a  tool  that  has  the  following 
features: 

Code  Input 
Command  Control 
Code  Instrumentation 
Complexity  Measurement 
Cross  Reference  Generation 
Dynamic  Coverage  Analysis 
Listing  Output 

Cross  Reference  and  Coverage  Reports  Output 
Instrumented  Code  Output 

would  have  the  following  classification  when  individual  keys 
are  chosen: 

I3.C1/T3.S3.S7.D3/U4. U5.M5 

Note  that  the  slash  is  used  to  separate  the  input, 
functional,  and  output  features  and  the  period  is  used  to 
separate  individual  keys.  It  can  be  seen  from  the  example 
that  the  key  clearly  and  succinctly  communicates  the  results 
of  classification  and  summarizes  the  tool  attributes. 

Since  the  identification  of  tool  features  can  be 
difficult  for  some  types  of  tools,  it  is  anticipated  that  a 
user's  guide  to  the  taxonomy  will  be  published.  This  guide 
will  provide  a  step-by-step  procedure  for  identifying  the 
features  of  tools  with  many  actual  examples  for  a  wide 
variety  of  software  development  tools. 

Comparison  o  f  Software  Tool s  and  Information  Retr  ieval .  In 
section  3,  we  described  a  user  who  was  interested  in  a  tool 
that  has  the  features  of  high  level  language  input,  program 
instrumentation,  dynamic  coverage  analysis,  and  output 
report  generation.  Using  individual  keys  from  the  taxonomy, 
a  retrieval   request  could  be  specified  as  follows: 

I3/T3&D3/U5 

where  13,  T3,  D3,  and  U5  represent  the  features,  the  slash 
is  used  to  separate  input  and  output,  and  the  &  is  a  logical 
'and'   of  the  features.     In  English     this     retrieval  request 
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would  be:  "Give  me  all  the  tools  which  accept  high  level 
language  programs  as  input,  perform  the  functions  of  program 
instrumentation  and  dynamic  coverage  analysis,  and  provide 
reports  as  output."  A  retrieval  of  this  request  from  the  NBS 


Software  Tools  Database  would  yield  the 
tools: 


following     list  of 


NBS  ANALYZER 

PET 

NODAL 

ITDEM 

COTUNE  II 

LOGIC 

FAVS 


JOVIAL  TCA 

RXVP 

PACE 

TEST  PREDICTOR 

TATTLE 

PDS 

CAVS 


EAVS 

DYNA 

PACE-C 

JIGSAW 

TPT 

JAVS 


After  examining  and  comparing  the  application  languages  and 
the  additional  features  offered  by  these  tools,  the 
retrieval  request  could  be  further  specified  as  follows: 

13  '  FORTRAN/T3&D3&  (S1'^S8)/U5 

where  SI  represents  code  auditing,  S8  represents  data  flow 
analysis,  the  '  delineates  the  actual  language  of  the  source 
code,  and  "  is  a  logical  'or'.  This  request  in  English 
would  be:  "Give  me  all  the  tools  which  accept  Fortran 
programs  as  input,  perform  the  functions  of  program 
instrumentation  and  dynamic  analysis  and  either  code 
auditing  or  data  flow  analysis,  and  provide  reports  as 
output."  This  retrieval  request  would  yield  the  following 
tool s : 


PET 


FAVS 


TPT 


Note  that  the  number  of  tools  that  a  potential 
must  consider  has  been  greatly  reduced. 


tool  user 


6.     Scope  of  the  Taxonomy 

The  taxonomy  is  primarily  a  classification  scheme  for 
software  development  tools.  There  are,  however,  many  tools 
that  are  broader  in  function  and  application  that  can  not  be 
easily  classified  by  the  taxonomy.  These  tools  include 
operating  systems,  system  utilities  (linkage  editors, 
loaders,  tape  handlers,  file  system,  sorters),  data  base 
management  systems,  and  management  information  systems. 
These  tools  serve  as  an  extended  part  of  a  system  and  are 
outside  the  scope  of  the  taxonomy.  Since  most  of  the 
features  in  the  taxonomy  are  specific  to  the  software 
development  process,  only  tools  specific  to  software 
development  should  be  classified. 
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7.  Conclusion 

Software  tools  are  powerful  productivity  and  quality 
aids  that  are  not  being  effectively  used  in  many  Federal 
programming  environments.  This  report  has  discussed  an 
effort  to  lessen  this  problem  by  providing  a  formal  way  in 
which  tools  can  be  classified.  The  use  of  a  taxonomy  of 
tool  features  can  be  used  to  (1)  categorize  currently 
available  tools,  (2)  standardize  terminology  associated  with 
tools,  and  (3)  ease  the  task  of  comparing  and  evaluating 
tools.  Use  of  the  taxonomy  should  help  users  determine 
precisely  what  a  given  tool  can  offer  and  consequently  what 
its  benefits  are. 
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